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DETAILED ACTION 

Response to Amendment 

1 . The Examiner acknowledges the amended claims filed on October 6, 
2009. Claim 60 has been amended. Claims 1-25, 29, 36, 39 and 49-56 have 
been cancelled. Claims 63 and 34 have been newly added. 

Response to Arguments 

2. Applicant's arguments filed October 6, 2009 have been fully considered 
but they are not persuasive. 

3. The Applicant argues the following: 

a. "The Office Action states that Denyar '438 teaches this limitation at 
page 12, lines 10-19 where Denyar '438 states that the "color channels 
are interpolated to provide missing data by low-pass-filtering using a 
square area template" and that "the component colour channel represents 
a lowpass-filtered (spatially smoothed) version of the ideal component 
colour channel response." (Office Action, page 8). Applicants respectfully 
disagree with the Office Action's conclusion because Denyar '438 does 
not teach or suggest what data is used from the square area template to 
conduct the interpolation or obtain a spatially smoothed version of the 
ideal component colour channel response. Using a green pixel as an 
example, as suggested by the Office Action at page 8, Denyar '438 does 
not teach or suggest estimating the level of the red or blue light received 
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by a green pixel based on a measurement of the red or blue light obtained 
by a red or blue pixel, and instead only describes "using a square area 
template". (Denvar '438). Therefore. Denvar '438 does not teach or 
suggest estimating "the level of the first spectral component in the light 
received by at least one of the second photosensitive sites based on at 
least one measurement of the first spectral component obtained 
respectively by at least one of the first photosensitive sites" as recited by 
claim 26 ". 

The Office Action also states that Denyar '438 "discloses that for 
luminance determination, the Green color component is calculated 
performing interpolation of Green color components surrounding sites that 
do not receive Green color to obtain the luminance component for said 
sites" at page 13, line 8 -page 15, line 20 of Denyar '438. (Office Action, 
page 8). Applicants respectfully disagree with this statement. At page 13, 
line 8 - page 15, line 20, Denyar '438 only discusses estimating 
luminance, and does not disclose estimating the level of a spectral 
component of light. Therefore the portion of Denyar cited by the Office 
Action is irrelevant to the current rejection of claim 26. 
> The Examiner disagrees. It is noted from page 12, lines 10-37, that 
Denyer discloses calculating the values of the green colour on a site that 
is not green (i.e. Red or Blue). This can also be shown in fig. 4, wherein a 
pixel is verified whether is a green pixel or not, if is not a green pixels, the 
values of the green pixels surrounding the target pixel are averaged (or 
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interpolated) to obtain the corresponding green value. Furthermore, 
Denyer discloses tin page 12, If 26-32 that the green pixels are elected to 
be "luminance-sensing" pixels and Green pixel signals are taken to be 
representative of luminance as well as being "representative of Green 
colour. Thus, even if in Denyer, the luminance value is calculated, the 
values of the green pixels are used to estimate the luminance value or 
green value on a red or blue pixel (Note that the Examiner interpreted the 
first spectral component as Green and the Second spectral component as 
Blue, therefore, Denyer discloses estimating the level of the first spectral 
component (Green) in the light received by at least one of the second 
photosensitive sites (capturing either red or blue) based on at least one 
measurement of the first spectral component obtained respectively by at 
least one of the first photosensitive sites as claimed). It is also noted that 
in page 13, line 8 -page 15, line 20, specifically in page 15, lines 1-20, 
Denyer recites " The estimated luminance signal obtained for a non-Green 
pixel sited at the centre of both a row and a column of five pixels provides 
an estimated luminance signal for the fourth i.e. luminance channel of the 
camera . This signal then has subtracted from it a spatially smoothed 
version of the luminance-representative signal, this being the particular 
output of a 5x5 square area lowpass-filtering template corresponding to 
the colour used for luminance sensing (in this case Green). This 
technique, sometimes known as "un-sharp masking", obtains a signal 
which approximates to a highpass-filtered (i.e. "edge-extracted") 
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luminance signal. The "edge-extracted" signal is then superimposed on to 
the spatially-smoothed colour channel signals obtained over the same 5 x 
5 pixel area so as to produce a Red. Green. Blue (RGB) pixel triplet 
including full colour and luminance information . RGB pixel triplet signals 
obtained in this way enable the camera to produce a contour-enhanced 
image which is more resolved, and thus more pleasing, to the human eye, 
than the image obtained purely from the spatially smoothed colour 
channel signals." This further teaches that the spectral components are 
also calculated since Denyer teaches that the luminance components are 
applied to spatially-smoothed colour channels obtained over the same 5 x 
5 pixel area so as to produce a Red, Green, Blue (RGB) pixel triplet 
including full colour and luminance information. Also, Denyer discloses 
the use of a lowpass filter (28) to restore the colour values from the 5 x 5 
pixel area as shown in fig. 7. 

> The Examiner understands that the Denyer reference teaches all 
the limitations of claim 26 as presented. Therefore, the rejections are 
maintained. 
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Claim Rejections - 35 USC § 102 

4. The following is a quotation of the appropriate paragraphs of 35 
U.S.C. 102 that form the basis for the rejections under this section made in this 
Office action: 

A person shall be entitled to a patent unless - 

(a) the invention was known or used by others in this country, or patented or described in a printed 
publication in this or a foreign country, before the invention thereof by the applicant for a patent. 

5. Claims 26-28, 30-35, 37, 38, 40, 42, 58 and 63 are rejected under 35 
U.S.C. 102(a) as being anticipated by Denyer et al., WO 97/35438 A1. 

6. Regarding claim 26, Denyer et al. discloses an imager (See fig. 7), 
comprising: 

a semiconductor substrate (Denyer et al. discloses a semiconductor 
substrate by disclosing that the imager is part of a single chip camera and that 
the production process of the image sensor is well disposed to deposition of the 
color filters when the products are in silicon-wafer form. See fig. 7, page 16, 
lines 13-21); 

an array of photosensitive sites (Fig. 7: 23; see also fig. 1; page 11, lines 
29-37; page 12, lines 1-8) located on the substrate (Note that the sensor array 23 
is located on the single camera chip 20 as shown in fig. 7 (See page 18, lines 5- 
30; also page 12, lines 1-8). This teaches that the array of photosensitive sites is 
located on the substrate), the array including 

a plurality of first photosensitive sites (See fig. 1) having a plurality 
of first color filters (As shown in fig. 1, Denyer et al. discloses a checkerboard 
pattern filter for Red, Green, and Blue colors. For examination purposes, the 
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Examiner is interpreting the Green color filters as the plurality of first color filters) 
arranged above said first photosensitive sites to allow only a first spectral 
component of light to reach said first photosensitive sites (Denyeret al. discloses 
that the pixels are covered with color filters corresponding to desired spectral 
components. See page 12, lines 1-8. See also fig. 1, which shows the 
arrangement of Red, Green and Blue colors. This teaches that the first sensitive 
sites are covered with a plurality of first color filters (corresponding to green color 
as interpreted by the Examiner) to allow only a first spectral component (Green) 
of light to reach said first photosensitive sites), wherein each first photosensitive 
site comprises a configuration enabling each first photosensitive site to measure 
the level of a first spectral component in light received (Green color components) 
by the respective first photosensitive site (See page 12, lines 1-19), and 

a plurality of second photosensitive sites having a plurality of 
second color filters (As shown in fig. 1, Denyer et al. discloses a checkerboard 
pattern filter for Red, Green, and Blue colors. For examination purposes, the 
Examiner is interpreting the Blue color filters as the plurality of second color 
filters) arranged above said second photosensitive sites to allow only a second 
spectral component of light to reach said second photosensitive sites (as 
discussed above, Denyer et al. discloses that the pixels are covered with color 
filters corresponding to desired spectral components. See page 12, lines 1-8. 
See also fig. 1, which shows the arrangement of Red, Green and Blue colors. 
This teaches that the second sensitive sites are covered with a plurality of 
second color filters (corresponding to Red color as interpreted by the Examiner) 
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to allow only a first spectral component (Green) of light to reach said first 
photosensitive sites), wherein each second photosensitive site comprises a 
configuration enabling each second photosensitive site to measure the level of a 
second spectral component in light received (Red color components) by the 
respective second site {Seepage 12, lines 1-19), said second spectral 
component being different from said first spectral component (Note that the 
Examiner is interpreting the first color component as Green and the second color 
component as Red, which are different from each other); and 

an interpolator (See processor subsystem 25 as shown in fig. 7. 
Denyer et al. discloses performing interpolation on the color components using 
low-pass filter 28 and also teaches performing interpolation for obtaining the 
luminance components of the captured image data. See page 18, lines 5-30; 
see also page 12, lines 10-37; page 13, line 8 - page 14, line 10; page 15, lines 
1-20) located on the substrate (Denyer et al. discloses that the processor 
subsystem is located on the same chip as the array of photosensitive sites. See 
page 18, lines 5-30; also page 16, lines 13-21) and comprising a configuration 
enabling the interpolator to estimate the level of the first spectral component 
(Green color) in the light received by at least one of the second photosensitive 
sites (Photosensitive sites measuring a color other than Green (i.e. Red)) based 
on at least one measurement of the first spectral component obtained 
respectively by at least one of the first photosensitive sites (Denyer et al. 
discloses applying a low-pass filter (As shown in fig. 7: 28) to obtain the missing 
colors in a respective site. See page 12, lines 10-37. This teaches the use of at 
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least one measurement of the first spectral component (Green color) to estimate 
the level of the first spectral component (Green color) in the light received by at 
least one of the second photosensitive sites since Denyer et al. discloses that by 
performing low-pass filter to the color components, a spatially smoothed version 
of the ideal color components (in this case Green color) is obtained (See page 
12, lines 10-19). Denyer et al. also discloses that for luminance determination, 
the Green color component is calculated by performing interpolation of Green 
color components surrounding sites that do not receive Green color to obtain the 
luminance component for said sites (See figs. 3 and 4; page 13, line 8 - page 15, 
line 20)). 

7. Regarding claim 27, Denyer et al. discloses that the first spectral 
component is a primary color of light (As shown in fig. 1, Denyer et al. discloses a 
checkerboard pattern filter for Red, Green, and Blue colors. For examination 
purposes, the Examiner is interpreting the Green color filters as the first spectral 
component which is a primary color of light). 

8. Regarding claim 28, Denyer et al. discloses that each second 
photosensitive site comprises a configuration enabling each second 
photosensitive site to measure the level of a second spectral component in light 
(Red color component) received by the respective second photosensitive site 
(Since Denyer et al. discloses a checkerboard pattern filter for Red, Green, and 
Blue colors covering the array of photosensitive sites (See fig. 1). For 
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examination purposes, the Examiner is interpreting the light received by the 
photosensitive sites with the Red color filters as the second spectral component 
in light measured by the respective second photosensitive site), and 

the interpolator further comprises a configuration enabling the interpolator 
configured to estimate the level of the second spectral component {Red color) in 
the light received by at least one of the first photosensitive sites (Photosensitive 
sites measuring Green color) based on at least one measurement of the second 
spectral component (Red color) obtained respectively by at least one of the 
second photosensitive sites (As discussed in claim 26, Denyer et al. discloses 
applying a low-pass filter (As shown in fig. 7: 28) to obtain the missing colors in a 
respective site. See page 12, lines 10-37. This teaches the use of at least one 
measurement of the second spectral component (Red color) to estimate the level 
of the second spectral component (Red color) in the light received by at least one 
of the first photosensitive sites (Photosensitive sites measuring Green color) 
since Denyer et al. discloses that by performing low-pass filter to the color 
components, a spatially smoothed version of the ideal color components (in this 
case Red color) is obtained (See page 12, lines 10-19)). 

9. Regarding claim 30, Denyer et al. discloses a plurality of third 
photosensitive sites (Since Denyer et al. discloses a checkerboard pattern filter 
for Red, Green, and Blue colors covering the array of photosensitive sites (See 
fig. 1). For examination purposes, the Examiner is interpreting the photosensitive 
sites covered by a Blue color filter as third photosensitive sites), and 
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the interpolator further comprises a configuration enabling the interpolator 
to estimate the level of the first spectral component (Green color) in the light 
received by at least one of the third photosensitive sites (photosensitive sites 
measuring Blue color) based on at least one measurement of the first spectral 
component (Green color) obtained respectively by at least one of the first 
photosensitive sites (As discussed in claim 26, Denyer et al. discloses applying a 
low-pass filter (As shown in fig. 7: 28) to obtain the missing colors in a respective 
site. See page 12, lines 10-37. This teaches the use of at least one 
measurement of the first spectral component (Green color) to estimate the level 
of the first spectral component (Green color) in the light received by at least one 
of the third photosensitive sites (Photosensitive sites measuring Blue color) since 
Denyer et al. discloses that by performing low-pass filter to the color 
components, a spatially smoothed version of the ideal color components (in this 
case Green color) is obtained on the photosensitive sites measuring a different 
color (in this case Blue color) (See page 12, lines 10-19)), and to estimate the 
level of the second spectral component (Red color) in the light received by at 
least one of the third photosensitive sites (photosensitive sites measuring Blue 
color) based on at least one measurement of the second spectral component 
(Red color) obtained respectively by at least one of the second photosensitive 
sites (photosensitive sites measuring Red color) (As discussed in claim 26, 
Denyer et al. discloses applying a low-pass filter (As shown in fig. 7: 28) to obtain 
the missing colors in a respective site. See page 12, lines 10-37. This teaches 
the use of at least one measurement of the second spectral component (Red 
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color) to estimate the level of the second spectral component (Red color) in the 
light received by at least one of the third photosensitive sites (Photosensitive 
sites measuring Blue color) since Denyer et al. discloses that by performing low- 
pass filter to the color components, a spatially smoothed version of the ideal 
color components (in this case Red color) is obtained on the photosensitive sites 
measuring a different color (in this case Blue color) (See page 12, lines 10-19)). 



1 0. Regarding claim 31 , Denyer et al. discloses that each third 
photosensitive site has a plurality of third color filters {As shown in fig. 1, Denyer 
et al. discloses a checkerboard pattern filter for Red, Green, and Blue colors. For 
examination purposes, the Examiner is interpreting the Blue color filters as the 
plurality of third color filters) arranged above said third photosensitive sites to 
allow only a third spectral component of light to reach said third photosensitive 
sites (Denyer et al. discloses that the pixels are covered with color filters 
corresponding to desired spectral components. See page 12, lines 1-8. See 
also fig. 1, which shows the arrangement of Red, Green and Blue colors. This 
teaches that the third photosensitive sites are covered with a plurality of third 
color filters (corresponding to Blue color as interpreted by the Examiner) to allow 
only a third spectral component of light (Blue color) to reach said third 
photosensitive sites), and wherein each third photosensitive site comprises a 
configuration enabling each third photosensitive site to measure the level of a 
third spectral component (Blue color component) in light received by the 
respective third photosensitive site (Since Denyer et al. discloses a checkerboard 
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pattern filter for Red, Green, and Blue colors covering the array of photosensitive 
sites (See fig. 1). For examination purposes, the Examiner is interpreting the 
light received by the photosensitive sites with the Blue color filters as the third 
spectral component in light measured by the respective second photosensitive 
site), and 

the interpolator further comprises a configuration enabling the interpolator 
configured to estimate the level of the third spectral component (Blue color) in the 
light received by at least one of the first photosensitive sites (Photosensitive sites 
measuring Green color component) and/or at least one of the second 
photosensitive sites (Photosensitive sites measuring Red color component) 
based on at least one measurement of the third spectral component obtained 
respectively by at least one of the third photosensitive sites (As discussed in 
claim 26, Denyer et al. discloses applying a low-pass filter (As shown in fig. 7: 
28) to obtain the missing colors in a respective site. See page 12, lines 10-37. 
This teaches the use of at least one measurement of the third spectral 
component (Blue color) to estimate the level of the third spectral component 
(Blue color) in the light received by at least one of the first photosensitive sites 
(Photosensitive sites measuring Green color) and/or at least one of the second 
photosensitive sites (Photosensitive sites measuring Red color) since Denyer et 
al. discloses that by performing low-pass filter to the color components, a 
spatially smoothed version of the ideal color components (in this case Green 
color) is obtained on the photosensitive sites measuring a different color (in this 
case Green and Red color components) (See page 12, lines 10-19)). 
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1 1 . Regarding claim 32, Denyer et al. discloses that the first spectral 
component is a first primary color of light (As discussed in claims 26 and 27, as 
interpreted by the Examiner the first spectral component is the Green color which 
is the first primary color of light), 

the second spectral component is a second primary color of light (As 
discussed in claims 26 and 28, as interpreted by the Examiner the second 
spectral component is the Red color which is the second primary color of light), 
and 

the third spectral component is a third primary color of light (As discussed 
in claim 30, as interpreted by the Examiner the third spectral component is the 
Blue color which is the third primary color of light), Grounds for rejecting claims 
26-28, 30 and 31 apply here. 

12. Regarding claim 33, Denyer et al. discloses a line decoder (Readout 
circuit 24 as shown in fig. 7) located on the substrate (Note in fig. 7 that the 
readout circuit 24 is located on the same chip as the array of photosensitive sites 
23 and the interpolator 25 (processor subsystem)) and having at least one serial 
output for transferring out at least one line of measured spectral components 
from the array during a read out operation (Denyer et al. discloses that five lines 
of five pixels forming a block are read out from the array of photosensitive sites 
23 and output to an A/D converter (A/D block as shown in fig. 7), that would 
further output the digitized signals to a five-line memory (Fig. 7: 26) (See page 
18, lines 5-30). Therefore, Denyer et al. discloses that the line decoder 24 has at 
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least one serial output for transferring out at least one line of measured spectral 
components from the array during a read out operation as claimed); and 

an A/D conversion element (A/D block as shown in fig. 7) located on the 
substrate (Note that the A/D block is also part of the single chip camera as 
shown in fig. 7, therefore is also located on the substrate) and comprising a 
configuration enabling the A/D conversion element to receive the at least one line 
of measured spectral components read out from the line decoder (Denyer et al. 
discloses that the five lines of said five pixels forming the block output from the 
readout circuit 24 are received by the A/D converter (A/D as shown in fig. 7). 
See page 18, lines 5-30) and output the received measurements as digital values 
to the interpolator (Denyer et al. discloses that the A/D converter digitizes the 
received signals from the readout circuit 24 and output said digitized signals to 
the interpolation circuits (low pass filter 28 and block 30 (which perform 
interpolation for determining the luminance values)). See page 18, lines 10-19), 
and 

wherein the interpolator estimates the first spectral component levels in 
the second and third photosensitive sites (As discussed in claims 26 and 30, 
Denyer et al. discloses estimating the first spectral component levels (Green 
color) in the second (Photosensitive sites measuring Red color) and third 
(Photosensitive sites measuring Blue color) photosensitive sites), 

the second spectral component levels in the first and third photosensitive 
sites (As discussed in claims 28 and 30, Denyer et al. discloses estimating the 
second spectral component levels (Red color) in the first (Photosensitive sites 
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measuring Green color) and third (Photosensitive sites measuring Blue color) 
photosensitive sites), and 

the third spectral component level in the first and second photosensitive 
sites based on the digital values received from the A/D conversion element (As 
discussed in claim 31, Denyer et al. discloses estimating the third spectral 
component levels (Blue color) in the first (Photosensitive sites measuring Green 
color) and second (Photosensitive sites measuring Red color) photosensitive 
sites) based on the digital values received from the A/D conversion element (As 
shown in fig. 7, the interpolation performed by the low pass filter 28 and block 30 
(which perform interpolation for determining the luminance values), perform the 
color estimation based on the digital values output by the A/D converter (A/D 
block), which output the digital values to the 5-line memory 27 and the values 
stored in the 5-line memory are read out to the interpolation elements. See page 
18, lines 5-30). 

1 3. Regarding claim 34, Denyer et al. discloses a line decoder (Readout 
circuit 24 as shown in fig. 7) located on the substrate (Note in fig. 7 that the 
readout circuit 24 is located on the same chip as the array of photosensitive sites 
23 and the interpolator 25 (processor subsystem)) and having at least one serial 
output for transferring out at least one line of measured spectral components 
from the array during a read out operation (Denyer et al. discloses that five lines 
of five pixels forming a block are read out from the array of photosensitive sites 
23 and output to an A/D converter (A/D block as shown in fig. 7), that would 
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further output the digitized signals to a five-line memory (Fig. 7: 26) (See page 
18, lines 5-30). Therefore, Denyer et al. discloses that the line decoder 24 has at 
least one serial output for transferring out at least one line of measured spectral 
components from the array during a read out operation as claimed); and 

an A/D conversion element (A/D block as shown in fig. 7) located on the 
substrate (Note that the A/D block is also part of the single chip camera as 
shown in fig. 7, therefore is also located on the substrate) and comprising a 
configuration enabling the A/D conversion element to receive the at least one line 
of measured spectral components read out from the line decoder {Denyer et al. 
discloses that the five lines of said five pixels forming the block output from the 
readout circuit 24 are received by the A/D converter (A/D as shown in fig. 7). 
See page 18, lines 5-30) and output the received measurements as digital values 
to the interpolator (Denyer et al. discloses that the A/D converter digitizes the 
received signals from the readout circuit 24 and output said digitized signals to 
the interpolation circuits (low pass filter 28 and block 30 (which perform 
interpolation for determining the luminance values)). See page 18, lines 10-19), 
and 

wherein the interpolator estimates the first spectral component levels in 
the second photosensitive sites based on the digital values received from the A/D 
conversion element (As discussed in claims 26 and 30, Denyer et al. discloses 
estimating the first spectral component levels (Green color) in the second 
photosensitive sites (Photosensitive sites measuring Red color). See page 12, 
lines 10-19). 



Application/Control Number: 10/774,603 Page 18 

Art Unit: 2622 

14. Regarding claim 35, Denyer et al. discloses a line decoder (Readout 
circuit 24 as shown in fig. 7) located on the substrate (Note in fig. 7 that the 
readout circuit 24 is located on the same chip as the array of photosensitive sites 
23 and the interpolator 25 (processor subsystem)) and having at least one serial 
output for transferring out at least one line of measured spectral components 
from the array during a read out operation (Denyer et al. discloses that five lines 
of five pixels forming a block are read out from the array of photosensitive sites 
23 and output to an A/D converter (A/D block as shown in fig. 7), that would 
further output the digitized signals to a five-line memory (Fig. 7: 26) (See page 
18, lines 5-30). Therefore, Denyer et al. discloses that the line decoder 24 has at 
least one serial output for transferring out at least one line of measured spectral 
components from the array during a read out operation as claimed), wherein the 
at least one serial output of the line decoder transfers out either several 
sequential lines or a block of measured spectral components from the array 
during each read out operation (As discussed above, Denyer et al. discloses that 
five lines of five pixels forming a block are read out from the array of 
photosensitive sites 23 and output to the A/D converter. Therefore, Denyer et al. 
discloses that at least one serial output of the line decoder transfers out either 
several sequential lines (since the line decoder transmits data (five pixels per 
line) from five sequential lines) or a block of measured spectral components from 
the array during each read out operation (since the five lines output belong to a 
block in the array of photosensitive sites, the block being used to estimate the 
missing color values). See page 18, lines 5-30). 
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15. Regarding claim 37, Denyer et al. discloses an imager (See figs. 1 and 
7), comprising: 

a semiconductor substrate (Denyer et al. discloses a semiconductor 
substrate by disclosing that the imager is part of a single chip camera and that 
the production process of the image sensor is well disposed to deposition of the 
color filters when the products are in silicon-wafer form. See fig. 7, page 16, 
lines 13-21); 

a plurality of first photosensitive sites (Fig. 7: 23; see also fig. 1; page 11, 
lines 29-37; page 12, lines 1-8) located on the substrate (Note that the sensor 
array 23 is located on the single camera chip 20 as shown in fig. 7 (See page 18, 
lines 5-30; also page 12, lines 1-8). This teaches that the plurality of first 
photosensitive sites is located on the substrate), said plurality of first 
photosensitive sites (See fig. 1) having a plurality of first color filters (As shown in 
fig. 1, Denyer et al. discloses a checkerboard pattern filter for Red, Green, and 
Blue colors. For examination purposes, the Examiner is interpreting the Green 
color filters as the plurality of first color filters) arranged above said first 
photosensitive sites to allow only a first spectral component of light to reach said 
first photosensitive sites (Denyer et al. discloses that the pixels are covered with 
color filters corresponding to desired spectral components. See page 12, lines 1- 
8. See also fig. 1, which shows the arrangement of Red, Green and Blue colors. 
This teaches that the first sensitive sites are covered with a plurality of first color 
filters (corresponding to green color as interpreted by the Examiner) to allow only 
a first spectral component (Green) of light to reach said first photosensitive sites), 
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wherein each first photosensitive site comprises a configuration enabling each 
first photosensitive site to measure the level of a first spectral component (Green 
color components) in light received by the respective first photosensitive site 
(See page 12, lines 1-19); 

a plurality of second photosensitive sites (Fig. 7: 23; see also fig. 1; page 
11, lines 29-37; page 12, lines 1-8) located on the substrate (Note that the sensor 
array 23 is located on the single camera chip 20 as shown in fig. 7 (See page 18, 
lines 5-30; also page 12, lines 1-8). This teaches that the plurality of second 
photosensitive sites is located on the substrate), said plurality of second 
photosensitive sites having a plurality of second color filters (As shown in fig. 1, 
Denyer et al. discloses a checkerboard pattern filter for Red, Green, and Blue 
colors. For examination purposes, the Examiner is interpreting the Blue color 
filters as the plurality of second color filters) arranged above said second 
photosensitive sites to allow only a second spectral component of light to reach 
said second photosensitive sites (as discussed above, Denyer et al. discloses 
that the pixels are covered with color filters corresponding to desired spectral 
components. See page 12, lines 1-8. See also fig. 1, which shows the 
arrangement of Red, Green and Blue colors. This teaches that the second 
sensitive sites are covered with a plurality of second color filters (corresponding 
to Red color as interpreted by the Examiner) to allow only a first spectral 
component (Green) of light to reach said first photosensitive sites), wherein each 
second photosensitive site comprises a configuration enabling each second 
photosensitive site to measure the level of a second spectral component (Red 
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color component) in light received by the respective second photosensitive site, 
said second spectral component being different from said first spectral 
component (Note that the Examiner is interpreting the first color component as 
Green and the second color component as Red, which are different from each 
other); and 

an interpolator (See processor subsystem 25 as shown in fig. 7. Denyer 
et al. discloses performing interpolation on the color components using low-pass 
filter 28 and also teaches performing interpolation for obtaining the luminance 
components of the captured image data. See page 18, lines 5-30; see also page 
12, lines 10-37; page 13, line 8 -page 14, line 10; page 15, lines 1-20) located 
on the substrate (Denyer et al. discloses that the processor subsystem is located 
on the same chip as the array of photosensitive sites. See page 18, lines 5-30; 
also page 16, lines 13-21) and comprising a configuration enabling the 
interpolator to receive digital data (digitized by A/D converter (A/D block as 
shown in fig. 7)) representing the spectral component levels measured in the first 
photosensitive sites (photosensitive sites measuring Green color) and the second 
photosensitive sites (photosensitive sites measuring Red color) (As shown in fig. 
7, Denyer et al. discloses that the color signals are read from the array of 
photosensitive sites 23 by a readout circuit 24 and digitized by an A/D converter 
(A/D block), the digital signals stored in a 5-line memory 26 and then transmitted 
to the interpolator (Note that in the processing subsystem, the interpolation is 
performed by a low pass filter 28 and block 30 (which perform interpolation for 
determining the luminance values) (See page 18, lines 5-30). This teaches that 
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the interpolator receives digital representing the spectral component levels 
measured in the first photosensitive sites as claimed), and to estimate the level of 
the first spectral component (Green color) in the light received by at least one of 
the second photosensitive sites (Photosensitive sites measuring a color other 
than Green (i.e. Red)) based on at least one digitized measurement of the first 
spectral component obtained respectively by at least one of the first 
photosensitive sites (Denyer et al. discloses applying a low-pass filter (As shown 
in fig. 7: 28) to obtain the missing colors in a respective site. See page 12, lines 
10-37. This teaches the use of at least one digitized measurement of the first 
spectral component (Green color) to estimate the level of the first spectral 
component (Green color) in the light received by at least one of the second 
photosensitive sites since Denyer et al. discloses that by performing low-pass 
filter to the color components, a spatially smoothed version of the ideal color 
components (in this case Green color) is obtained (See page 12, lines 10-19). 
Denyer et al. also discloses that for luminance determination, the Green color 
component is calculated by performing interpolation of Green color components 
surrounding sites that do not receive Green color to obtain the luminance 
component for said sites (See figs. 3 and 4; page 13, line 8 - page 15, line 20; 
page 18, lines 5-30)). 

16. Regarding claim 38, Denyer et al. discloses a plurality of third 
photosensitive sites (Fig. 7: 23; see also fig. 1; page 11, lines 29-37; page 12, 
lines 1-8), said plurality of third photosensitive sites having a plurality of third 
color filters (As shown in fig. 1, Denyer et al. discloses a checkerboard pattern 
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filter for Red, Green, and Blue colors. For examination purposes, the Examiner 
is interpreting the Blue color filters as the plurality of third color filters) arranged 
above said third photosensitive sites to allow only a third spectral component of 
light to reach said third photosensitive sites (Denyer et al. discloses that the 
pixels are covered with color filters corresponding to desired spectral 
components. See page 12, lines 1-8. See also fig. 1, which shows the 
arrangement of Red, Green and Blue colors. This teaches that the third 
photosensitive sites are covered with a plurality of third color filters 
(corresponding to Blue color as interpreted by the Examiner) to allow only a third 
spectral component of light (Blue color) to reach said third photosensitive sites), 
wherein each third photosensitive site comprises a configuration enabling each 
third photosensitive site to measure the level of a third spectral component (Blue 
color component) in light received by the respective third photosensitive site 
(Since Denyer et al. discloses a checkerboard pattern filter for Red, Green, and 
Blue colors covering the array of photosensitive sites (See fig. 1). For 
examination purposes, the Examiner is interpreting the light received by the 
photosensitive sites with the Blue color filters as the third spectral component in 
light measured by the respective second photosensitive site), and 

wherein the interpolator further comprises a configuration enabling the 
interpolator to estimate: 

the level of the first spectral component (Green color) in the light 
received by at least one of the third photosensitive sites (photosensitive sites 
measuring Blue color) based on at least one digitized measurement of the first 
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spectral component (Green color being digitized by A/D converter as shown in 
fig. 7. See page 18, lines 5-30) obtained respectively by at least one of the first 
photosensitive sites (photosensitive sites measuring Green color) (As discussed 
in claim 37, Denyer et al. discloses applying a low-pass filter (As shown in fig. 7: 
28) to obtain the missing colors in a respective site. See page 12, lines 10-37. 
This teaches the use of at least one digitized measurement of the first spectral 
component (Green color digitized by the A/D converter as shown in fig. 7) to 
estimate the level of the first spectral component (Green color) in the light 
received by at least one of the third photosensitive sites (Photosensitive sites 
measuring Blue color) since Denyer et al. discloses that by performing low-pass 
filter to the color components, a spatially smoothed version of the ideal color 
components (in this case Green color) is obtained on the photosensitive sites 
measuring a different color (in this case Blue color) (See page 12, lines 10-19; 
page 18, lines 5-30)), 

the level of the second spectral component (Red color) in the light 
received by at least one of the third photosensitive sites (photosensitive sites 
measuring Blue color) based on at least one digitized measurement of the 
second spectral component (Red color being digitized by A/D converter as 
shown in fig. 7. See page 18, lines 5-30) obtained respectively by at least one of 
the second photosensitive sites (photosensitive sites measuring Red color) (As 
discussed in claim 37, Denyer et al. discloses applying a low-pass filter (As 
shown in fig. 7: 28) to obtain the missing colors in a respective site. See page 
12, lines 10-37. This teaches the use of at least one digitized measurement of 
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the second spectral component (Red color digitized by the A/D converter as 
shown in fig. 7) to estimate the level of the second spectral component (Red 
color) in the light received by at least one of the third photosensitive sites 
(Photosensitive sites measuring Blue color) since Denyer et al. discloses that by 
performing low-pass filter to the color components, a spatially smoothed version 
of the ideal color components (in this case Red color) is obtained on the 
photosensitive sites measuring a different color (in this case Blue color) (See 
page 12, lines 10-19; page 18, lines 5-30)), and 

the level of the third spectral component {Blue color) in the light 
received by at least one of the first photosensitive sites (photosensitive sites 
measuring Green color) and/or at least one of the second photosensitive sites 
(photosensitive sites measuring Red color) based on at least one digitized 
measurement of the third spectral component (Blue color being digitized by A/D 
converter as shown in fig. 7. See page 18, lines 5-30) obtained respectively by 
at least one of the third photosensitive sites (photosensitive sites measuring Blue 
color) (As discussed in claim 37, Denyer et al. discloses applying a low-pass filter 
(As shown in fig. 7: 28) to obtain the missing colors in a respective site. See 
page 12, lines 10-37. This teaches the use of at least one digitized 
measurement of the third spectral component (Blue color digitized by the A/D 
converter as shown in fig. 7) to estimate the level of the third spectral component 
(Blue color) in the light received by at least one of the first photosensitive sites 
(Photosensitive sites measuring Green color) and/or at least one of the second 
photosensitive sites (Photosensitive sites measuring Red color) since Denyer et 
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al. discloses that by performing low-pass filter to the color components, a 
spatially smoothed version of the ideal color components (in this case Green 
color) is obtained on the photosensitive sites measuring a different color (in this 
case Green and Red color components) (See page 12, lines 10-19; page 18, 
lines 5-30)). 

1 7. Regarding claim 40, Denyer et al. discloses that the first spectral 
component is a first primary color of light (As discussed in claims 37 and 38, as 
interpreted by the Examiner the first spectral component is the Green color which 
is the first primary color of light), 

the second spectral component is a second primary color of light (As 
discussed in claims 37 and 38, as interpreted by the Examiner the second 
spectral component is the Red color which is the second primary color of light), 
and 

the third spectral component is a third primary color of light (As discussed 
in claims 37 and 38, as interpreted by the Examiner the third spectral component 
is the Blue color which is the third primary color of light), Grounds for rejecting 
claims 37 and 38 apply here. 

18. Regarding claim 42, Denyer et al. discloses that the interpolator includes 
at least one serial register (Denyer et al. discloses a plurality of five-pixel 
registers (Fig. 7: 27)) for storing digital bit values representing the spectral 
component measurements from a photosensitive site being interpolated and the 



Application/Control Number: 10/774,603 Page 
Art Unit: 2622 

photosensitive sites neighboring the photosensitive site being interpolated 
(Denyer et al. discloses that a 5X5 block is read out from the memory by 5-pixel 
registers 27, the block including a photosensitive site being interpolated and the 
photosensitive sites neighboring the photosensitive site being interpolated. See 
page 12, lines 10-19; page 18, lines 5-30). 

1 9. Regarding claim 58, Denyer et al. discloses that the interpolator further 
comprises a configuration enabling the interpolator to estimate the level of the 
first spectral component (Green color) in the light received by at least one of the 
second photosensitive sites (photosensitive sites measuring Red color) based on 
a measurement of the first spectral component (Green color) obtained 
respectively by only a plurality of the first photosensitive sites (Denyer et al. 
discloses that the interpolator estimates the level of the first spectral (Green 
color) component in the light received by at least one of the second 
photosensitive sites (photosensitive sites measuring Red color) based on a 
measurement of the first spectral component (Green color) obtained respectively 
a plurality of the first photosensitive sites by teaching that the missing colors are 
estimated by applying a lowpass filter to the surrounding photosensitive sites of 
the color being estimated (See page 12, lines 10-19)). 

20. Regarding claim 63, Denyer et al. discloses that the interpolator outputs 
a signal for the at least one of the second photosensitive sites (i.e. for receiving 
blue or red color spectral) that represents light received by the at least one of the 
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second photosensitive sites associated second color filter (i.e. blue), the signal 
comprising the estimated level of the first spectral component (green) of light and 
the measured level of the second spectral component of light (As discussed in 
claim 26, Denyer discloses that the green color values (first spectral component) 
are used to calculate the green spectral component on a second photosite (i.e. 
for receiving blue or red color spectral). Furthermore, as shown in fig. 7, the tree 
color values are outputted from the interpolator (see final output after adder 32 
wherein the three color values are outputted. This teaches that the interpolator 
would output both the estimated value as well as the color value detected by the 
photodiode. See page 12, lines 10-37; page 13, line 8 -page 15, line 20)). 



Claim Rejections - 35 USC § 103 

21 . The following is a quotation of 35 U.S.C. 1 03(a) which forms the basis for 
all obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described 
as set forth in section 1 02 of this title, if the differences between the subject matter sought to 
be patented and the prior art are such that the subject matter as a whole would have been 
obvious at the time the invention was made to a person having ordinary skill in the art to which 
said subject matter pertains. Patentability shall not be negatived by the manner in which the 
invention was made. 

22. Claims 41, 57 and 64 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Denyer et al., WO 97/35438 A1 in view of Acharya, US 
Patent 6,091,851. 

23. Regarding claim 41 , Denyer et al. does not explicitly disclose that the 
interpolator output twenty four bits of color data for each photosensitive site, with 



each color value being represented by eight bits. 
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However, Acharya teaches the concept of performing color recovery of 
imager captured by a camera (Fig. 3: 330) using a single sensor having a Bayer 
pattern color filter array in order to obtain a full resolution image from an object 
(Fig. 3: 340) being photographed, wherein the individual color components of 
each pixel area represented by eight bits (in order to represent a color intensity 
range from 0-255) and the pixels of the image after interpolation is performed 
would have a total resolution of twenty four bits (Col. 1, lines 4-48; col. 2, lines 
40-52; col. 3, lines 16-41; col. 5, line 26 - col. 6, line 26; col. 9, lines 1-8). 
Acharya also discloses that the interpolation method can be performed by 
hardware and firmware and that the interpolation method can be performed by 
the camera processor (Fig. 3: 32) (Col. 10, lines 7-17). 

Therefore, taking the combined teaching of Denyer et al. in view of 
Acharya as a whole, one of an ordinary skill in the art, after appreciating the 
advantages of the interpolation method of Acharya, would find obvious at the 
time the invention was made to modify the imager of Denyer et al. by having the 
interpolator output twenty four bits of color data for each photosensitive site, with 
each color value being represented by eight bits. The motivation to do so would 
have been to have a desirable amount of color intensity values (256 color 
intensity values) for each color of each pixel in the image and to better represent 
luminance in recovering missing color components to have the output image to 
better resemble the original color of the object prior to its image capture. 
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24. Regarding claim 57, Although Denyer et al. discloses that the interpolator 
estimates the level of the first spectral (Green color) component in the light 
received by at least one of the second photosensitive sites (photosensitive sites 
measuring Red color) based on a measurement of the first spectral component 
(Green color) obtained respectively a plurality of the first photosensitive sites by 
teaching that the missing colors are estimated by applying a lowpass filter to the 
surrounding photosensitive sites of the color being estimated (See page 12, lines 
10-19), Denyer et al. does not explicitly disclose that the estimation is based on 
only two of the first photosensitive sites. 

However, Acharya teaches the concept of performing color interpolation 
to obtain missing color components on a captured image and discuses the use of 
only two pixels for estimating color components as a simple approach (Acharya 
teaches that for a missing Red color component, the two nearest Red color pixel 
value would be averaged to estimate said missing Red color component, and the 
same would be performed for Blue and Green color components to obtain an 
approximation of the original object's true color. See col. 1, line 49 - col. 2, line 
7). 

Therefore, taking the combined teaching of Denyer et al. in view of 
Acharya as a whole, it would have been obvious to one of an ordinary skill in the 
art at the time the invention was made to apply the concept of using only two 
color components to approximate an object's true color component missing at a 
particular pixel location as taught by Acharya to modify the interpolator of Denyer 
to estimate the level of the first spectral component (Green color) in the light 
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received by at least one of the second photosensitive sites (photosensitive sites 
measuring Red color) based on a measurement of the first spectral component 
{Green color) obtained respectively by only two of the first photosensitive sites. 
The motivation to do so would have been to calculate the missing color 
components using a simple interpolation method that would reduce the time for 
estimating the missing color components as an alternative to using a 5 x 5 pixel 
kernel, thus allowing increasing the speed of the interpolation process. 

Regarding claim 64, Denyer et al. does not explicitly disclose that the 
interpolator estimates the level of the first spectral component in the light 
received by the at least one of the second photosensitive sites based on 
measurements of the first spectral component obtained respectively by at least 
two of the first photosensitive sites located in a same row as the at least one of 
the second photosensitive sites. However, Acharya teaches the concept of 
performing color recovery of imager captured by a camera {Fig. 3: 330) using a 
single sensor having a Bayer pattern color filter array in order to obtain a full 
resolution image from an object {Fig. 3: 340) being photographed, wherein the 
color components of a particular pixel (i.e. blue pixel in position (6, 3) as shown in 
fig. 2) the values of two adjacent green pixels in the row and the values of two 
adjacent pixels in a column are used to calculate the green color on said blue 
pixel (Co/. 1, lines 4-48; col. 2, lines 40-52; col. 3, lines 16-41; col. 5, line 26 - 
col. 6, line 26; col. 9, lines 1-8). Acharya also discloses that the interpolation 
method can be performed by hardware and firmware and that the interpolation 
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method can be performed by the camera processor (Fig. 3: 32) (Col. 10, lines 7- 
17). 

Therefore, taking the combined teaching of Denyer et al. in view of 
Acharya as a whole, one of an ordinary skill in the art, after appreciating the 
advantages of the interpolation method of Acharya, would find obvious at the 
time the invention was made to modify the imager of Denyer et al. by having the 
interpolator using measurements of the first spectral component obtained 
respectively by at least two of the first photosensitive sites located in a same row 
as the at least one of the second photosensitive sites as an alternative to the 
method in Denyer. The motivation to do so would have been to use an 
alternative method to calculate the colors not present in a particular photosite 
using the color values that would contribute the most into approximating the real 
color for a predetermined target pixel. 

25. Claims 43 and 59-62 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Denyer et al., WO 97/35438 A1 in view of Sakurai et al., 
US Patent 5,990,946. 

26. Regarding claim 43, Denyer et al. does not explicitly disclose that for 
estimating a spectral component level for a photosensitive site, the interpolator 
digitally weights the values of the spectral component being estimated, as 
measured by the photosensitive sites providing the measurements and which are 
currently stored in the at least one serial register, based on the distances of the 
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photosensitive sites providing the measurements from the photosensitive site for 
which the spectral component is being estimated. 

However, Sakurai et al. teaches a digital camera (See figs. 12 and 14), 
comprising an imager (See fig. 12: 104 and fig. 14: 90) for capturing an image; 
an A/D converter (Fig. 12: 107) for digitizing the color signals received from the 
imager (Col. 7, lines 21-60) to be memorized in a memory (254 as shown in fig. 
12; see also memory 91 as shown in fig.14) and a color interpolation circuit 
(Color interpolation circuit 3 as shown in figs. 14 and 15) for estimating the color 
components missing on each photosensitive site (Col. 8, line 17 -col. 9, line 26). 
Sakurai et al. further teaches that the color interpolation circuit comprises five 
multipliers (See fig. 17: 12-16) for weighting the values of the spectral component 
being estimated, as measured by the photosensitive sites providing the 
measurements and which are currently stored in at least one serial register 
(Sakurai et al. discloses using a plurality of delay units (Fig. 17: 8-11) for storing 
the spectral components being estimated (See col. 10, lines 9-27). Since the 
delay units are used to hold the spectral components being estimated for further 
performing interpolation to said spectral components, the Examiner is reading the 
delay units 8-11 as a serial register since the spectral components measured, 
which are received from the memory, are shifted from each of the delay units to 
be output to the multipliers 8-11 for weighting the spectral components. See col. 
10, lines 9-27), based on the distances of the photosensitive sites providing the 
measurements from the photosensitive site for which the spectral component is 
being estimated (In col. 9, line 48 - col. 10, line 6, Sakurai et al. teaches a 
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plurality of equations that are applied to the spectral components by the 
interpolation circuit. It is noted that a lower weighting coefficient is applied to 
spectral components far from the target pixel to be estimated. For example the 
estimation for an n2 line (See lines in the portion of fig. 16 below), would be 
defined as: 

Mg= 1/2(1 /2Mg 1 2+ 1/2Mg14)+1/2(1/4Mg3 1 + 1/2Mg33+ 1/4Mg35) 

G= 1/2(1 74G1 1 + 1/2G13+ 1/4G1 5)+1/2(1/2G32+ 1/2G34) 

Ye=1/2(1/2Ye22+Ye24) 

Cy= 1/2(1 /2Cy2 1 +Cy23+ 1/2Cy25), 
It is noted that Mg31 and Mg35 are receiving a lower weighting coefficient (1/4), 
which is assigned based on the distance from the target pixel since said pixels 
are far from the target pixel and pixels that are closer to the target pixel are 
receiving a larger weighting coefficient ( 1/2). The same occurs for other 
estimations in the same line as well as other lines.). 
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Applying a weighting coefficient that is associated to the distance from the 
target pixel is advantageous because it would provide a smooth (natural) 
continuity for the image formed of the filtered spectral components. 
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Therefore, taking the combined teaching of Denyer et al. in view of 
Sakurai et al. as a whole, it would have been obvious to one of an ordinary skill in 
the art at the time the invention was made to apply the concept of weighting the 
values of the spectral component being estimated, which are currently stored in a 
serial register, based on the distances of the photosensitive sites providing the 
measurements from the photosensitive site for which the spectral component is 
being estimated as taught by Sakurai et al. to modify the teaching of Denyer et 
al. to have the interpolator digitally weighting the values of the spectral 
component being estimated, as measured by the photosensitive sites providing 
the measurements and which are currently stored in the at least one serial 
register, based on the distances of the photosensitive sites providing the 
measurements from the photosensitive site for which the spectral component is 
being estimated. The motivation to do so would have been to provide a smooth 
(natural) continuity for the image formed of the filtered spectral components. 

Regarding claims 59-62, Denyer et al. discloses the interpolator 
comprises at least one serial register {Denyer et al. discloses a plurality of five- 
pixel registers (Fig. 7: 27)) for storing digital bit values representing the spectral 
component measurements from a photosensitive site being interpolated and a 
plurality photosensitive sites neighboring the photosensitive site (24 
photosensitive sites neighboring the photosensitive site being interpolated in the 
5x5 block) being interpolated (Denyer et al. discloses that the color signals are 
read from the array of photosensitive sites 23 by a readout circuit 24 and 
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digitized by an A/D converter (A/D block), the digital signals stored in a 5-line 
memory 26 and then transmitted to the interpolator (Note that in the processing 
subsystem, the interpolation is performed by a low pass filter 28 and block 30 
(which perform interpolation for determining the luminance values) (See page 18, 
lines 5-30). This teaches that the register stores digital bit values representing 
the spectral component measurements from a photosensitive site being 
interpolated and the photosensitive sites neighboring the photosensitive site 
being interpolated). Denyer et al. does not explicitly disclose the plurality of 
photosensitive sites stored in the at least one serial register are four 
photosensitive sites; 

that the interpolator further comprises five scalar multipliers for multiplying 
the digital bit values of the spectral component measurements from the 
photosensitive site being interpolated and the four photosensitive sites 
neighboring the photosensitive site being interpolated; 

a first adder for adding the digital bit value of a first of the four 
photosensitive sites neighboring the photosensitive site being interpolated to a 
second of the four photosensitive sites neighboring the photosensitive site and a 
second adder for adding the digital bit value of a third of the four photosensitive 
sites neighboring the photosensitive site being interpolated to a fourth of the four 
photosensitive sites neighboring the photosensitive site; and 

a first dividing circuit for dividing in half a summation of the first and 
second of the four photosensitive sites neighboring the photosensitive site being 
interpolated and a second dividing circuit for dividing in half a summation of the 
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third and fourth of the four photosensitive sites neighboring the photosensitive 
site being interpolated. 

However, Sakurai et al. teaches a digital camera (See figs. 12 and 14), 
comprising an imager (See fig. 12: 104 and fig. 14: 90) for capturing an image; 
an A/D converter (Fig. 12: 107) for digitizing the color signals received from the 
imager (Col. 7, lines 21-60) to be memorized in a memory (254 as shown in fig. 
12; see also memory 91 as shown in fig.14) and a color interpolation circuit 
(Color interpolation circuit 3 as shown in figs. 14 and 15) for estimating the color 
components missing on each photosensitive site (Col. 8, line 17 -col. 9, line 26). 
Sakurai et al. further teaches using a plurality of delay units (Fig. 17: 8-11) for 
shifting five spectral components, a color component being interpolated and four 
color components neighboring the color component being estimated (See col. 10, 
lines 9-27; Since the delay units are used to hold the spectral component being 
estimated and the four spectral components neighboring the spectral component 
being estimated for further performing interpolation to said spectral components, 
the Examiner is reading the delay units 8-11 as a serial register since the 
spectral components measured, which are received from the memory, are shifted 
from each of the delay units to be output to a plurality of multipliers 8-11 for 
weighting the spectral components. See col. 10, lines 9-41). Sakurai et al. 
further teaches that the color interpolation circuit further comprises a plurality of 
interpolation filters (5-7 as shown in fig. 17) each five scalar multipliers 
(Coefficient setting units 12-16 as shown in fig. 17) for multiplying the digital bit 
values of the spectral component measurements from the photosensitive site 
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being interpolated [it is noted that the interpolation color filters 5-7 contain the 
same structure for interpolating missing components (See col. 10, lines 9-41) and 
that the color component that is being interpolated would receive a weight value 
when used for estimating the same spectral component missing at a different 
location since the output of said interpolation color filters 5-7 are further used to 
obtain the values of the spectral components missing in the obtained image data 
to satisfy the equations shown in col. 9, line 47 - col. 10, line 6 (See also col. 10, 
lines 9-41). Therefore, Sakurai et al. teaches that a scalar multiplier multiplies 
the digital bit values of the spectral component measurements from the 
photosensitive site being interpolated) and the four photosensitive sites 
neighboring the photosensitive site being interpolated (Sakurai et al. teaches that 
the coefficient setting units 12-16 are used to apply a weighting value to the 
spectral components being interpolated. Coefficient setting units 12 and 16 
apply a 1/4 weighting value and coefficient setting units 13-15 apply a 1/2 
weighting value to the spectral components being interpolated. See fig. 17; see 
also col. 9, line 27- col. 10, line 36. As discussed above, it is noted that Sakurai 
et al. teaches combining the outputs of the interpolation filters to yield a particular 
color component. As shown in fig. 7, the color components can be obtained from 
four values of the spectral components. For example, to satisfy the equation for 
estimating a Ye color component, 

Ye = 1 A ( 1 AYe42 + 1 AYe44) + 1 A (AYe62 + 1 AYe64), it is noted that four values of 
the Ye spectral components have been assigned a particular weight value (A) 
and have been interpolated to estimated the Ye component missing at position 
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G53 (line n5 as shown in fig. 16). This also teaches that the estimation for the 
different color components may be performed by the use of different amount of 
spectral components measured by the photosensitive sites and furthermore, 
Sakurai et al. further teaches that the estimation bay be performed by using 
either one-dimensional filter, two-dimensional filters and while the block as 
taught corresponds to lines of 5 pixels, the lines may be of 3 or 7 pixels (See 
col. 10, lines 33-41). This suggests the use of any desired amount of 
photosensitive site values for estimating the missing color components in the 
image)); 

a first adder (See fig. 17: 17) for adding the digital bit value of a first of the 
four photosensitive sites (corresponding to spectral component being weighted 
by coefficient setting unit 12 as shown in fig. 17) neighboring the photosensitive 
site being interpolated to a second of the four photosensitive sites {corresponding 
to spectral component being weighted by coefficient setting unit 16 as shown in 
fig. 17) neighboring the photosensitive site and a second adder (See fig. 17: 18) 
for adding the digital bit value of a third of the four photosensitive sites 
(corresponding to spectral component being weighted by coefficient setting unit 
13 as shown in fig. 17) neighboring the photosensitive site being interpolated to a 
fourth of the four photosensitive sites (corresponding to spectral component 
being weighted by coefficient setting unit 15 as shown in fig. 17) neighboring the 
photosensitive site (See fig. 17; see also col. 9, line 27- col. 10, line 3); and 

a first dividing circuit (Coefficient setting unit 19 as shown in fig. 17) for 
dividing in half a summation of the first and second of the four photosensitive 
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sites {corresponding to spectral components being weighted by coefficient setting 
units 12 and 16 as shown in fig. 17) neighboring the photosensitive site being 
interpolated and a second dividing circuit (Coefficient setting unit 20 as shown in 
fig. 17) for dividing in half a summation of the third and fourth of the four 
photosensitive sites {corresponding to spectral components being weighted by 
coefficient setting units 13 and 15 as shown in fig. 17) neighboring the 
photosensitive site being interpolated. 

Therefore, taking the combined teaching of Denyer et al. in view of 
Sakurai et al., after acknowledging the teaching of estimating the missing color 
components in an image by using the values a plurality spectral components 
being weighted by a plurality of multipliers, added and divided by half and that 
the estimation may be perform by different amounts of values of a spectral 
component, including four values to obtain the missing color component at a 
particular location as taught by Sakurai et al., it would have been obvious to one 
of an ordinary skill in the art at the time the invention was made to modify the 
interpolator of Denyer et al. to have the plurality of photosensitive sites stored in 
the at least one serial register are four photosensitive sites; to have the 
interpolator with a structure comprising five scalar multipliers for multiplying the 
digital bit values of the spectral component measurements from the 
photosensitive site being interpolated and the four photosensitive sites 
neighboring the photosensitive site being interpolated; a first adder for adding the 
digital bit value of a first of the four photosensitive sites neighboring the 
photosensitive site being interpolated to a second of the four photosensitive sites 
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neighboring the photosensitive site and a second adder for adding the digital bit 
value of a third of the four photosensitive sites neighboring the photosensitive 
site being interpolated to a fourth of the four photosensitive sites neighboring the 
photosensitive site; and a first dividing circuit for dividing in half a summation of 
the first and second of the four photosensitive sites neighboring the 
photosensitive site being interpolated and a second dividing circuit for dividing in 
half a summation of the third and fourth of the four photosensitive sites 
neighboring the photosensitive site being interpolated. The motivation to do so 
would have been to provide an alternative structure to the interpolator in Denyer 
et al. to perform the estimation of the missing color components with the purpose 
of producing a smooth (natural) continuity for the image formed of the filtered 
spectral components. 

27. Claims 44-48 are rejected under 35 U.S.C. 103(a) as being 
unpatentable over Denyer et al., WO 97/35438 A1 in view of Denyer et al., 
WO 97/20434 A1 (thereinafter referred as Denyer et al. '434). 

28. Regarding claim 44, Denyer et al. discloses an imaging device (See figs. 
1 and 7), comprising: 

an imager (See fig. 7: 23) which comprises 

a substrate (Denyer et al. discloses a semiconductor substrate by 
disclosing that the imager is part of a single chip camera and that the production 
process of the image sensor is well disposed to deposition of the color filters 
when the products are in silicon-wafer form. See fig. 7, page 16, lines 13-21); 
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an M x N array of photosensitive sites (See fig. 1; see also fig. 7: 
23) located on the substrate (Note that the sensor array 23 is located on the 
single camera chip 20 as shown in fig. 7 (See page 18, lines 5-30; also page 12, 
lines 1-8), the array including 

a plurality of first photosensitive sites {Fig. 7: 23; see also fig. 
1; page 11, lines 29-37; page 12, lines 1-8) located on the substrate (Note that 
the sensor array 23 is located on the single camera chip 20 as shown in fig. 7 
(See page 18, lines 5-30; also page 12, lines 1-8). This teaches that the plurality 
of first photosensitive sites is located on the substrate), said plurality of first 
photosensitive sites (See fig. 1) having a plurality of first color filters (As shown in 
fig. 1, Denyer et al. discloses a checkerboard pattern filter for Red, Green, and 
Blue colors. For examination purposes, the Examiner is interpreting the Green 
color filters as the plurality of first color filters) arranged above said first 
photosensitive sites to allow only a first spectral component of light to reach said 
first photosensitive sites (Denyer et al. discloses that the pixels are covered with 
color filters corresponding to desired spectral components. See page 12, lines 1- 
8. See also fig. 1, which shows the arrangement of Red, Green and Blue colors. 
This teaches that the first sensitive sites are covered with a plurality of first color 
filters (corresponding to green color as interpreted by the Examiner) to allow only 
a first spectral component (Green) of light to reach said first photosensitive sites), 
wherein each first photosensitive site comprises a configuration enabling each 
first photosensitive site to measure the level of a first spectral component (Green 
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color components) in light received by the respective first photosensitive site 
(See page 12, lines 1-19); 

a plurality of second photosensitive sites {Fig. 7: 23; see also 
fig. 1; page 11, lines 29-37; page 12, lines 1-8) located on the substrate (Note 
that the sensor array 23 is located on the single camera chip 20 as shown in fig. 
7 (See page 18, lines 5-30; also page 12, lines 1-8). This teaches that the 
plurality of second photosensitive sites is located on the substrate), said plurality 
of second photosensitive sites having a plurality of second color filters (As shown 
in fig. 1, Denyer et al. discloses a checkerboard pattern filter for Red, Green, and 
Blue colors. For examination purposes, the Examiner is interpreting the Blue 
color filters as the plurality of second color filters) arranged above said second 
photosensitive sites to allow only a second spectral component of light to reach 
said second photosensitive sites (as discussed above, Denyer et al. discloses 
that the pixels are covered with color filters corresponding to desired spectral 
components. See page 12, lines 1-8. See also fig. 1, which shows the 
arrangement of Red, Green and Blue colors. This teaches that the second 
sensitive sites are covered with a plurality of second color filters (corresponding 
to Red color as interpreted by the Examiner) to allow only a first spectral 
component (Green) of light to reach said first photosensitive sites), wherein each 
second photosensitive site comprises a configuration enabling each second 
photosensitive site to measure the level of a second spectral component (Red 
color component) in light received by the respective second photosensitive site, 
said second spectral component being different from said first spectral 
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component (Note that the Examiner is interpreting the first color component as 
Green and the second color component as Red, which are different from each 
other); and 

an interpolator (See processor subsystem 25 as shown in fig. 7. 
Denyer et al. discloses performing interpolation on the color components using 
low-pass filter 28 and also teaches performing interpolation for obtaining the 
luminance components of the captured image data. See page 18, lines 5-30; 
see also page 12, lines 10-37; page 13, line 8 - page 14, line 10; page 15, lines 
1-20) located on the substrate (Denyer et al. discloses that the processor 
subsystem is located on the same chip as the array of photosensitive sites. See 
page 18, lines 5-30; also page 16, lines 13-21) and comprising a configuration to 
receive digitized color component values (digitized by AID converter {AID block 
as shown in fig. 7)) corresponding to the measurements obtained in the first 
photosensitive sites (photosensitive sites measuring Green color) and the second 
photosensitive sites (photosensitive sites measuring Red color) (As shown in fig. 
7, Denyer et al. discloses that the color signals are read from the array of 
photosensitive sites 23 by a readout circuit 24 and digitized by an A/D converter 
(A/D block), the digital signals stored in a 5-line memory 26 and then transmitted 
to the interpolator (Note that in the processing subsystem, the interpolation is 
performed by a low pass filter 28 and block 30 (which perform interpolation for 
determining the luminance values) (See page 18, lines 5-30). This teaches that 
the interpolator receives digital representing the spectral component levels 
measured in the first photosensitive sites as claimed), to estimate the level of the 
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first color component (Green color) in the light received by at least one of the 
second photosensitive sites (Photosensitive sites measuring a color other than 
Green (i.e. Red)) based on at least one digitized color component obtained 
respectively by at least one of the first photosensitive sites (Denyer et al. 
discloses applying a low-pass filter (As shown in fig. 7: 28) to obtain the missing 
colors in a respective site. See page 12, lines 10-37. This teaches the use of at 
least one digitized measurement of the first spectral component (Green color) to 
estimate the level of the first spectral component (Green color) in the light 
received by at least one of the second photosensitive sites since Denyer et al. 
discloses that by performing low-pass filter to the color components, a spatially 
smoothed version of the ideal color components (in this case Green color) is 
obtained (See page 12, lines 10-19). Denyer et al. also discloses that for 
luminance determination, the Green color component is calculated by performing 
interpolation of Green color components surrounding sites that do not receive 
Green color to obtain the luminance component for said sites (See figs. 3 and 4; 
page 13, line 8 - page 15, line 20; page 18, lines 5-30)) and to estimate the level 
of the second color component (Red color) in the light received by at least one of 
the first photosensitive sites (Photosensitive sites measuring Green color) based 
on at least one digitized color component obtained respectively from at least one 
of the second photosensitive sites (Photosensitive sites measuring Red color) 
(As discussed above, Denyer et al. discloses applying a low-pass filter (As 
shown in fig. 7: 28) to obtain the missing colors in a respective site. See page 
12, lines 10-37. This teaches the use of at least one digitized second color 
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component (Red color digitized byA/D converter as shown in fig. 7) to estimate 
the level of the second color component (Red color) in the light received by at 
least one of the first photosensitive sites (Photosensitive sites measuring Green 
color) since Denyer et al. discloses that by performing low-pass filter to the color 
components, a spatially smoothed version of the ideal color components (in this 
case Red color) is obtained (See page 12, lines 10-19)). 

Denyer et al. does not explicitly disclose a display for displaying an image 
on an array of M x N pixels. 

However, Denyer et al. '434 discloses an imaging device {See figs. 3 and 
4), comprising: 

a display (Fig. 4: 30) for displaying an image on an array of M x N pixels 
(page 11, line 33 - page 12, line 25); and 

an imager (Fig. 3: 1 and 4: 1) which comprises 

a substrate (by teaching that the imaging array is located in a chip, 
Denyer et al. discloses a semiconductor substrate; see page 13, lines 1-22), 

an M x N array of photosensitive sites located on the substrate (fig. 
3 shows an M x N array of pixels 2), the array including 

a plurality of first photosensitive sites located in the substrate 
(the array in Denyer et al. is divided into three kind of photosensitive sites (red, 
green and blue); see fig. 2; page 10, line 23 - page 12, line 32), wherein each 
first photosensitive site is configured to measure the level of a first color (i.e. 
green) component in light received by the respective first photosensitive site, and 
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a plurality of second photosensitive sites (the array in 
Denyer et al. is divided into three kind of photosensitive sites (red, green and 
blue); see fig. 2; page 10, line 23 - page 12, line 32) located in the substrate, 
wherein each second photosensitive site is configured to measure the level of a 
second color component (i.e. red color) in light received by the respective second 
photosensitive site (taking in consideration green, red and blue as a first, second 
and third spectral components for examining purposes, the second spectral 
components can be read as red. Denyer et al. discloses measuring red, green 
and blue colors as discussed in claim 26 above), said second spectral 
component being different from said first spectral component (the array in Denyer 
et al. is divided into three kind of photosensitive sites (red, green and blue); see 
fig. 2; page 10, line 23 - page 12, line 32. This teaches that the second spectral 
component being different from said first spectral component as claimed since 
each photosensitive site is receiving a single color spectral of the three colors 
that the full array receives); and 

an interpolator (Denyer et al. discloses a processing unit (Fig. 4: 
28) to perform color interpolation to the red, green and blue signals to form 
synchronous, parallel color channel signals for the video signal before being 
output to a display unit (Fig. 4: 30); page 11, line 33 - page 12, line 25) located 
on the substrate (Denyer et al. discloses that the processing unit can be 
incorporated in the same chip, where the imaging array is located; page 13, lines 
1-22) and configured to receive digitized color component values (output from an 
A/D converter 26 as shown in fig. 4; page 11, line 33 - page 13, line 23) 
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corresponding to the measurements obtained in the first and second 
photosensitive sites, to estimate the level of the color component in the light 
received by at least one of the second photosensitive sites based on at least one 
digitized color component value obtained respectively from at least one of the 
other photosensitive sites, and to estimate the level of the second color 
component in the light received by at least one of the other photosensitive sites 
based on at least one digitized color component value obtained respectively from 
at least one of other photosensitive sites (Denyer et al. discloses reconstructing 
the image colors of each pixels by performing interpolation to obtain an RGB 
value for each pixel location) (Page 10, line 23 - page 13, line 22). 

Having an imaging device including a display for displaying an image on 
an array of M x N pixels is advantageous because it improve the portability of the 
imaging device by allowing the user to review the captured image data on the 
same device. 

Therefore, taking the combined teaching of Denyer et al. in view of Denyer 
et al. '434 as a whole, it would have been obvious to one of an ordinary skill in 
the art at the time the invention was made to apply the concept of having an 
imaging device with a display for displaying the images captured by said imaging 
device as taught in Denyer et al. '434 to modify the imaging device of Denyer et 
al. to include a display for displaying an image on an array of M x N pixels. The 
motivation to do so would have been to improve the portability of the imaging 
device by allowing the user to review the captured image data on the same 
device. 
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29. Regarding claim 45, the combined teaching of Denyer et al. in view of 
Denyer et al. '434 further teaches that the interpolator estimates the color 
component level not measured in each respective photosensitive site in at least 
one line of photosensitive sites in the array during a readout operation (Denyer et 
al. discloses that five lines of five pixels forming a block are read out from the 
array of photosensitive sites 23 and output to an A/D converter (A/D block as 
shown in fig. 7), that would further output the digitized signals to a five-line 
memory (Fig. 7: 26) and the subjected to color interpolation (using lowpass filter 
28 and block 30 (which perform interpolation for determining the luminance 
values) to obtain the missing colors in a respective site) (See page 18, lines 5- 
30). Therefore, Denyer et al. discloses that the interpolator estimates the color 
component level not measured in each respective photosensitive site in at least 
one line of photosensitive sites in the array during a readout operation since the 
block in Denyer et al is formed by a plurality of lines). 

30. Regarding claim 46, the combined teaching of Denyer et al. in view of 
Denyer et al. '434 further teaches that the interpolator estimates the color 
component level not measured in each respective photosensitive site in several 
sequential lines of photosensitive sites in the array during a readout operation 
(Denyer et al. discloses that five lines of five pixels forming a block are read out 
from the array of photosensitive sites 23 and output to an A/D converter (A/D 
block as shown in fig. 7), that would further output the digitized signals to a five- 
line memory (Fig. 7: 26) and the subjected to color interpolation (using lowpass 
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filter 28 and block 30 (which perform interpolation for determining the luminance 
values) to obtain the missing colors in a respective site) (See page 18, lines 5- 
30). Therefore, Denyer et al. discloses that the interpolator estimates the color 
component level not measured in each respective photosensitive site in several 
sequential lines of photosensitive sites in the array during a readout operation 
since the block in Denyer et al is formed by a plurality of sequential lines). 

31 . Regarding claim 47, the combined teaching of Denyer et al. in view of 
Denyer et al. '434 further teaches that the interpolator estimates the color 
component level not measured in each respective photosensitive site in a block 
of photosensitive sites in the array during a readout operation (Denyer et al. 
discloses that five lines of five pixels forming a block are read out from the array 
of photosensitive sites 23 and output to an A/D converter (A/D block as shown in 
fig. 7), that would further output the digitized signals to a five-line memory (Fig. 7: 
26) and the subjected to color interpolation (using lowpass filter 28 and block 30 
(which perform interpolation for determining the luminance values) to obtain the 
missing colors in a respective site) (See page 18, lines 5-30). Therefore, Denyer 
et al. discloses that the interpolator estimates the color component level not 
measured in each respective photosensitive site in a block of photosensitive sites 
in the array during a readout operation). 

32. Regarding claim 48, the combined teaching of Denyer et al. in view of 
Denyer et al. '434 further teaches a plurality of third photosensitive sites {Fig. 7: 
23; see also fig. 1; page 11, lines 29-37; page 12, lines 1-8), said plurality of third 
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photosensitive sites having a plurality of third color filters (As shown in fig. 1, 
Denyer et al. discloses a checkerboard pattern filter for Red, Green, and Blue 
colors. For examination purposes, the Examiner is interpreting the Blue color 
filters as the plurality of third color filters) arranged above said third 
photosensitive sites to allow only a third spectral component of light to reach said 
third photosensitive sites (Denyer et al. discloses that the pixels are covered with 
color filters corresponding to desired spectral components. See page 12, lines 1- 
8. See also fig. 1, which shows the arrangement of Red, Green and Blue colors. 
This teaches that the third photosensitive sites are covered with a plurality of 
third color filters (corresponding to Blue color as interpreted by the Examiner) to 
allow only a third spectral component of light (Blue color) to reach said third 
photosensitive sites), wherein each third photosensitive site comprises a 
configuration enabling each third photosensitive site to measure the level of a 
third spectral component (Blue color component) in light received by the 
respective third photosensitive site (Since Denyer etal. discloses a checkerboard 
pattern filter for Red, Green, and Blue colors covering the array of photosensitive 
sites (See fig. 1). For examination purposes, the Examiner is interpreting the 
light received by the photosensitive sites with the Blue color filters as the third 
spectral component in light measured by the respective second photosensitive 
site), and 

wherein the interpolator further comprises a configuration enabling the 
interpolator to estimate: 
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the level of the first color component {Green color) in the light 
received by at least one of the third photosensitive sites (photosensitive sites 
measuring Blue color) based on at least one digitized measurement of the first 
color component (Green color being digitized byA/D converter as shown in fig. 7. 
See page 18, lines 5-30) obtained respectively by at least one of the first 
photosensitive sites (photosensitive sites measuring Green color) (As discussed 
in claim 37, Denyer et al. discloses applying a low-pass filter (As shown in fig. 7: 
28) to obtain the missing colors in a respective site. See page 12, lines 10-37. 
This teaches the use of at least one digitized measurement of the first color 
component (Green color digitized by the A/D converter as shown in fig. 7) to 
estimate the level of the first color component (Green color) in the light received 
by at least one of the third photosensitive sites (Photosensitive sites measuring 
Blue color) since Denyer et al. discloses that by performing low-pass filter to the 
color components, a spatially smoothed version of the ideal color components (in 
this case Green color) is obtained on the photosensitive sites measuring a 
different color (in this case Blue color) (See page 12, lines 10-19; page 18, lines 
5-30)), 

the level of the second color component (Red color) in the light 
received by at least one of the third photosensitive sites (photosensitive sites 
measuring Blue color) based on at least one digitized measurement of the 
second color component (Red color being digitized by A/D converter as shown in 
fig. 7. See page 18, lines 5-30) obtained respectively by at least one of the 
second photosensitive sites (photosensitive sites measuring Red color) (As 
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discussed in claim 37, Denyer et al. discloses applying a low-pass filter (As 
shown in fig. 7: 28) to obtain the missing colors in a respective site. See page 
12, lines 10-37. This teaches the use of at least one digitized measurement of 
the second color component (Red color digitized by the A/D converter as shown 
in fig. 7) to estimate the level of the second color component (Red color) in the 
light received by at least one of the third photosensitive sites (Photosensitive 
sites measuring Blue color) since Denyer et al. discloses that by performing low- 
pass filter to the color components, a spatially smoothed version of the ideal 
color components (in this case Red color) is obtained on the photosensitive sites 
measuring a different color (in this case Blue color) (See page 12, lines 10-19; 
page 18, lines 5-30)), and 

the level of the third color component (Blue color) in the light 
received by at least one of the first photosensitive sites {photosensitive sites 
measuring Green color) and/or at least one of the second photosensitive sites 
(photosensitive sites measuring Red color) based on at least one digitized 
measurement of the third color component (Blue color being digitized by A/D 
converter as shown in fig. 7. See page 18, lines 5-30) obtained respectively by 
at least one of the third photosensitive sites (photosensitive sites measuring Blue 
color) (As discussed in claim 37, Denyer et al. discloses applying a low-pass filter 
(As shown in fig. 7: 28) to obtain the missing colors in a respective site. See 
page 12, lines 10-37. This teaches the use of at least one digitized 
measurement of the third color component (Blue color digitized by the A/D 
converter as shown in fig. 7) to estimate the level of the third color component 
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(Blue color) in the light received by at least one of the first photosensitive sites 
(Photosensitive sites measuring Green color) and/or at least one of the second 
photosensitive sites (Photosensitive sites measuring Red color) since Denyer et 
al. discloses that by performing low-pass filter to the color components, a 
spatially smoothed version of the ideal color components (in this case Green 
color) is obtained on the photosensitive sites measuring a different color (in this 
case Green and Red color components) (See page 12, lines 10-19; page 18, 
lines 5-30)). 



Conclusion 

33. THIS ACTION IS MADE FINAL. Applicant is reminded of the extension of 
time policy as set forth in 37 CFR 1 .1 36(a). 

A shortened statutory period for reply to this final action is set to expire 
THREE MONTHS from the mailing date of this action. In the event a first reply is 
filed within TWO MONTHS of the mailing date of this final action and the advisory 
action is not mailed until after the end of the THREE-MONTH shortened statutory 
period, then the shortened statutory period will expire on the date the advisory 
action is mailed, and any extension fee pursuant to 37 CFR 1.136(a) will be 
calculated from the mailing date of the advisory action. In no event, however, will 
the statutory period for reply expire later than SIX MONTHS from the mailing 
date of this final action. 
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